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"UNCLASSIFIED 


ESTIMATING  MEAN  RELIABILITY  GROWTH 

1 .    Introduction 

A  problem  of  considerable  importance  in  current  reliability  studies  is 
that  of  accounting  for  changes  in  reliability  that  result  from  various 
actions  designed  to  modify  a  part  or  a  system.    Such  modifications  may 
range  from  design  changes  in  the  early  states  of  development  to  corrective 
action  taken  to  remove  modes  of  failure  that  have  been  ^served  in  a  test- 
ing  program  at  a  later  stage  of  development.    Any  moda4  Jn  which  it  is 
assumed  that  such  modifications  never  decrease  the  ^reliability  (probaba- 
bility  of  successful  operation)  has  come  to  be  titled  "reliability  growth." 

Despite  the  importance  of  the  problem  and  the  interest  in  solutions, 
very  little  published  research  on  the  topic  of  reliability  growth  exists .  A 
brief  list  of  some  papers  on  the  subject  is  given  in  the  bibliography.  In 
one  reference  [3]  the  writer,  along  with  others,  has  developed  one  model 
to  account  for  reliability  growth  and  several  estimations  problems  are  dis- 
cussed there.    This  same  model  is  the  main  concern  of  the  present  report 
and  is  repeated  here  for  the  sake  of  completeness  . 

Suppose  that  an  item  or  a  system  is  to  be  tested  arifl  bach  test  may 
result  in  success  with  probability  p    or  exactly  one  of   K   fixed,  but 
otherwise  unspecified,  modes  of  failure.    The  parameter   p      is  referred 
to  as  the  initial  reliability  and  we  denote  the  probability  of  failure  of  type 

K 

i  by   q.    for  i  =  l,  2,   . . . ,  K.    Thus,  p    +  S  q   =  1    and  if  we  assume  that 
i  o     i=1    i 

N    fixed,  mutually  independent  tests  are  to  be  performed,  the  underlying   ; 
probability  model  is  that  of  the  multinomial  distribution  with  parameters 
N,  p   ,  q, ,   . . .,  q    .    Accordingly,  we  denote  the  number  of  observed 

OIK 

successes  in    N    tests  by   N    while  N.  is  used  to  denote  the  number  of 

o  1 

observed  failures  of  type    i.    Thus,  N    ,  N, ,   . . . ,  NT,  are  (marginally) 

o       1  K 

.;  N 

binomial  random  variables  subject  to  the  restriction     £  N.  -    N  t   j 

;  i=o     '    ' 

l-  '■"      '  ; 


By  fixing    K   in  the  preceding  formulation  it  is  tacitly  assumed  that 
no  new  failure  modes  are  ever  introduced.    Also,  it  is  Assumed  that  no 
corrective  action  is  to  be  taken  until  all   N   tests  are  performed  so  that 
our   procedures  will  be  based  on  fixed  sample  size  experiments .  Having 
performed    N   tests,  it  is  assumed  that  each  observed  failure  mode  can  be 
classified  as    to  type  and  that  ah  attempt  is  then  made  to  remove  that  mode 
of  failure.    However,  it  is  not  assumed  that  a  failure  mode  is  necessarily    . 
removed  once  it  is  observed.    Indeed,  subject  to  experience  with  the  item  . 
or  system,  it  is  assumed  that  there  is  a  known  probability  a   of  removing 
the  i     mode  of  failure  given  that  corrective  action  is  taken  which  in  turn 
is  always  taken  if  and  only  if  the  failure  mode  is  observed. 

Under  the  above  model,  there  are  at  least  three  measures  of  current 

reliability  that  are  of  interest  each  being  appropriate  at  possibly  different 

stages  of  development.     First,  and  the  most  natural,  is  the  actual  current 

reliability,  say   R  ,  which  exists  after  the  tests  and  after,  the  corrective 

action  takes  place.    Prior  to  testing,    R   is  a  random  variable  of  course  and 

may  be  written  as  follows,    Let  y   =  1  if  N    >  0  and  0  otherwise,  so  that  y 

is  a  random  variable  that  accounts  for  whether  or  not  the  i      failure  mode 

occurs.    Also,  let   x   =  1    if  corrective  action  on  the  i      failure  mode  actually 

1  -  ;I 

removes  that  mode  of  failure  and    0    otherwise.    We  may  then  write , 


(1.1),     R-    pQ    +   Ixjft 
i=l 

In  this  form  we  see  that  current  reliability  is  the  initial  reliability  plus 
any  failure  probability  that  has  been  observed  and  actually  removed. 
Reliability  is  thus  not  increased  by  a  particular  mode  if  either  it  is  not 
observed  or  it  is  observed  but  not  removed . 


:2 
is 


,  i 
.    ..)    i    \ 


The  quantity    R   which  we  have  defined  above  is  of  primary  interest 

f  |  ! 

after  the  complete  testing  program  when  the  actual  current  pliability  is 

desired.    One  perfectly  straightforward  way  of  estimating  R  is  to  perform 

■  t  i 

N  additional  tests,  observing   N        successes  and  use  the  usual  success 

•      •  °  ■..■';,.••:•) 

ratio    N  /N    as  an  estimate.    It  this  is  feasible,  such  a  procedure  certainly 
o 

is  on  safe  grounds  statistically  speaking.    If,  however,  the  cost  or  availa- 
bility of  items  prohibits  this  direct  approach,  it  is  necessary  to  adopt  a 
cruder  measure  of  current  reliability  and  use  the  results  oT  the    N    tests  to 
draw  inference  about  the  amount  of  reliability  growth .    One  such  measure 

would  be  the  conditional  mean  of   R  ,  conditioned  on  the  observed  values 

* 
of  NJ#  i  =  0,  1,   ...,  K.    This  conditional  mean,  denoted  p     ,  is  derived  in 

[3]  and  is  given  by 

K 

* 


(1.2)    pQ     =    pQ   +  ^a^ 


N  1=1 

where  y.    is  defined  above .    Such  an  average  has  an  advantage  in  that 

averaging  is  taken  with  respect  to  whether  or  not  corrective  action  is  sue*? 

cessful  as  a  function  only  of  the  failure  modes  that  are  observed.    In  this 

sense,  p       is  not  the  true  current  reliability,  that  is  success  probability, 

as  it  is  erroneously  referred  to  in  [2]  but  is  already  an  "average"  relia- 

* 
bility.    The  analysis  of  both    R    and    p      is  the  main  concern  of  Report  No.  2  ;-. 

of  this  study  [5]  and  will  not  be  discussed  further  in  this  report. 

A  third  measure  of  curorent  reliability  is  the  unconditional  mean    of  the 
current  reliability  (which  is  the  same  as  the  expected  value  of  p      )  and  is 
an  "average"  taken  over  all  possible  outcomes  of  the  experiment.  This 
measure  of  reliability  is  relevant  before  testing  and  before  corrective  action . 
Such  a  quantity  would  be  suitable  for  assessing  the  potential  gain  in  relia- 
bility to  be    derived  from  a  corrective  action  program .    Being  a  true  parameter 
in  the  strict  sense  of  the  word,  it  lends  itself  quite  well  to  standard  statis- 
tical estimation  tools  .    It  is  shown  in  [3]  that  this  mean  reliability,  denoted 
(j, ,  is  given  by  the  formula , 


K  K  r 

(i.3)  n-  po  +Xvi'-  Iaiqi(1-qi)N 

i=l  i-l 

References  [2]  and  [3]  address  themselves  mainly  to  the  problem  of  find- 
ing estimator!  for   p,    under  the  conditions  stated  in  the  above  model . 
Several  estim^Jors  are  defined  in  both  references  and  various  properties 
(of  both  a  positive  and  a  negative  nature)  are  discussed.    Three  of  these  . 
are  of  interest  in  the  present  study  and,  for  the  sake  of  definiteness ,  the 
notation  of  [3]  (which  does  not  always  agree  with  that  of  [2]  even  for  the 
same  quantity)  will  be  adopted . 

The  maximum  likelihood  estimator  for   \i    is  denoted    p_    and  is  defined 
quite  simply  by, 

N         i=t        N        .    .        N 
i=l 

An  exact  expression  for  the  expected  value  of   p.    was  not  available  in 
[3]  although  approximate  expressions  were  derived.    This  created  some 
limitations  in  comparing    p     with  other  estimators  on  an  equitable  basis  . 

0 

One  of  the  purposes  of  the  present  study  is  to  resolve  this  problem  of 
exact  expressions  for  the  moments  .    The  results  are  presented  in  the 
sections  to  follow . 

Another  estimator,  derived  in  [3]  to  meet  the  requirements  of  unbiased- 
ness  ,  is  denoted    p      and  is- given  by  the  expression, 

i=ii  =1 . 

ponding  bias  of   p^  , 
simply  by 


The  corresponding  bias  of   pc ,  defined  by  b(p_)    =   E(p_)-|j.  is  given  quite 


K     '       N+l 
(1.6)    b(p  )  *    (-1)N     £   a^ 

i=l 


It  should  be  noted  that    p      is  not  unbiased  and,  indeed,  it  is  shown  in 

o 

[3]  that  no  unbiased  estimator  for   p.    exists.    However,  the  bias  for    p 
may  be  so  small  as  to  be  negligible  and  this  will  be  verified  in  Section  3 
to  follow . 

For  reasons  peculiar  to  the  Navy,  a  third  estimator  for   \x    has  been 

adopted  by  Special  Projects  and  is  extensively  discussed  in  [2] .    This 

estimator  is  here  (and  in  [3])  denoted  by   p,.    and  is  defined  by, 

b 

K  r~ 

N_         ^        N,  I  ai    if   Nj>    1 


(1.7)    p.    =  — 9    +  V  z    — i  \ 

6         m  A     i    M 


N  L    i    N  i      I  0     otherwise 

i=l 

The  moments  of   p      are  easily  computed  and  the  bias  is  accordingly  given 
b 

by 

K 

3s,,       ,  N-l 


(1.8)    b(p6)  =    -'£  a^     (1-q^ 
i=l 


It  should  be  observed  that  the  bias  of    p_    is  always  negative  so  that    p. 

b  b 

is  a  "conservative"  estimator.    However,  it  may  be  that  the  amount  of  bias 
is  serious  enough  to  discredit  conservatism  in  some  cases.    Several  samples , 
admitting  a  wide  variety  of  choice  for  the  various  parameters  in  the  above 
model,  are  delineated  in  [2]  .    However,  a  simulated  version  of  the  random 
variable  p       is  used  as  a  reference  point  rather  than   p.  ,  no  results  regard- 
ing p_  are  presented  and,  for  reasons  mentioned  previously,  the  moments 
o 

of  p,,  are  omitted  from  discussions  .  An  examination  of  the  behavior  of  all 
tif  these  elements  for  the  same  examples  constitutes  another  portion  of  the 
present  study.    Results  are  summarized  in  the  following  sections. 

2  .    Maximum  Likelihood  Estimator 

One  of  the  common  features  of  the  two  estimators    p,.    and    p.    defined 

5  6 

in  Section  1  is  that  no  credit  is  given  to  failure  modes  that  occur  only  once . 
This  is  easily  seen  by  examining  equations  1J5  and  1.7  where,  if   N  =0 
for  any  given  i,  that  term  involving    N    vanishes  and  therefore  does  not 
affect  the  value  of  the  estimator.    For  small  sample  sizes,  this  is  somewhat 

5    '     • 


undesirable  since  such  a  procedure  appears  to  ignore  some  of  the  infor- 
mation in  the  sample.  This  may  be  the  price  one  pays  fpr  attempting  to 
avoid  overestimating    p,    which  was  a  requirement  constantly  kept  in  mind 

in  deriving    p_    and  p    .    It  should  be  noted  that  the  maximum  likelihood 
o  b 

estimator  does  not  have  this  particular  feature  and  every  occurrence  of  a 
failure  mode  is  allowed  to  increase  the  estimate  of   \x   .    To  see  how  the 

bias  is  affected  we  first  compute  the  expected  value  of   p    . 

i  3 

N       N       N      '"       .  Nk 

Since  (l--)    =,  7       (k    )<"»    "J     '    we  may  write 

*  ..... 

i5!  i=l  N         k=0  Nk    , 

K        N  k+1 

-J- 1  F..^C^  • 

N        i*i     k=l  N 


Thus ,  from  linearity, 

K        N  k+1 

i*l     k=l  JyjH+1 

But  for    i=l,  2:,    . . ./  K,    N" '.is  binomial  with  parameters    q     and    N    so 
that,  for  each   K=l,2  ,   ... ,  N, 

V     k+i\      r  .k+i  r  N\    i  N"j 

E(Ni  )  =  Z j   ux^ 

j=o 

Substituting  in  the  preceding  expression, 

■\  .  *j  *   rNY,NV  &  &fc  $■>♦» 
e(p.)  =  p„-t   y  y  a  k ?  a.  -j  q  (i-qt)  ^j  j 


=   po  + 


K       N  u  ,     N  k+1 

i-i       j=o  k=l 


"  k+l  N 


k=l 

K        N 


HI     J=l 
Also,     i   [  N;^  =   fN-1^    for   j   ^  0    so  we  may  write, 

JM     v-  j  -^  S.J  -1  y 

b<p3)  ■  p0  +  Z  Z  "i  <£!)  ^0-ON"'0-<¥)N) 

i=l     j=l     . 

=  Po +  i^zcwo-^r-  z  i^-v^To&S 

i-il    j=l  i=l      j=l 

After  some  minor  simplications ,  we  finally    write 

(2.i,  e<p3) =Po  +f  $■  -l  Y^cn^T'^f 

i«l  i=l    j=0 

A 

The  bias  of   p      is  then  immediate  and  may  be  written  as , 
o 

K  ^j         N-2  ...1  n 

(2.2)    b,P3)  =  I  .ftO-^     [1  ^^i$<X^tm)  ] 

1=1  j=o 

Before  examining  the  magnitude  of  the  bias  b(p  J  in  comparison  with  other 
estimators,  it  is  possible  to  provide  an  alternative  form  of  (2  .1)  using 


Stirling  numbers  of  the  second  kind.    This  form  may  be  more  appropriate 
for  some  purposes  than  the  one  given.    As  before,  finding    E(p  )  reduces 
to  being  able  to  write  the  general  m        moment    of  the  binomial  distri- 
bution explicitly.    The  expression  of  the  latter  in  terms  of  Stirling  numbers 
does  not  seem  to  be  widely  recorded  and  is  stated  below  as  a  theorem  for 

the  record.    With  regard  to  the  proof,  we  follow  the  notation  of  [4]  so 

(r)  k  I 

that,  for  positive  integers    k    and    r,  k     =  k(k-l) (k-r+1)  rVr^Tj  if 

k  ^  r  and  is  zero  if   k  <  r;  V     denotes  the  r      Stirlings<number  of  the 
second  kind  of  order   k  .  .*&' 

Theorem     Let    X   be  binomial  with^  parameters    n    and    p. 

m  Dm 

Then,    E  (a)  =  )    n    .  0       p     for  every  positive  integer,  m. 

Note:'   If    m>n,    E(Xm)  =  Yn(r)  ^  pr  since    ri      =  0    if   r  >  n  . 

r=l, 

Proof:  Let    m    be  any  positive  integer.    By  definition, 

n 


t{f)  =KOpkqn'k  km- 


k=l 
m 


m 
But,    k"'  ■  ,}   y.r    kx"       so    that  , 

r=l 


=    }    «}0r    k  so 


n    m  ,-kJmkw 


n       m  a 


k=l     r=l 

(r) 
Since    k        =    0  if   k  <  r  we  may  simplify  further  to, 

m      n 

r=l    k=r 

™  n-r 

=  1 n  J m  pr  2  C"r  i p" q  n"r" 


r=l 


a=0 


m        (Jm 

=  ?   n(V     P  q.e.d. 

L.  ■   r 

r=l 

'"     ) 
To  write  E  (p  )  in  terms  of  Stirling  numbers  we  expand (1 .4)  to  obtain, 

»3  ■  if +  I'ai-I  - 1  ai N'  (N-N')N  •» that-  ' ' 

i  =  l      ;  1=1  nN+1 

K  K 

(2  .3)    E  <p3)  =    pQ    ♦  I  a.c.    -   I  JJ-    -E  [  N.  (n-AJ1  ] 

1=1  1=1  N 

Now  1st    Z.  =    N.  (N  -  NJN    and    Y.    =    N-N.      so  that 
i  l  i  l  l 

V 

Zt    =    (N-Yj  Y,N  =    NY.N  -  Y.N+1  ,    Since    N,  is  binomial  with  parameters 
i  i      i  ii  l  •. 

N  and  q.,  it  follows  that   Y,    is  binomial  with  parameters    N    and  1-q,  .    By 

l  i  ■  j  ■  *  i 

the  theorem , 

N 


E(Z1)=NE(Y1N)-E(YiN«)'-    ^    ^Jr  :     - 

i=l ' 
N  nN  0N+1 


r=l 
,N+1        /jN 


m        =C^     +  T<d>      (see' [4]  p.  17)    we  dan  simplify  to  Stirling 


Since 

numbers  of  the  same  order  whence, 


N       -N 

N 


E(Z.)    =    2  NW(i-Qj)     [<N-i)<2r  "WU]   | 


r=l 


Substituting  in  (2  .3)  we  finally  obtain  , 

K  K         N  Qi  •  ij  N  N 

(2.4)    E  (p  )  =  pQ    +   ^  -   ^         £4^  N(r)(l^qJ[(N-r)^r  "<^1  ] 

i=l  i=i       r=l        •  - 


Returning  to  the  bias  of    p      given  in  (2.2)  it  is  difficult,  because  of  the 
complexity  of  the  expression,  to  make  general  statements.    Certainly  p. 
may  both  overestimate  as  well  as  underestimate  p,  .    A  simple  example 
shows  this.    Suppose  N=2 ,  K=l,  and  a  =1 .    Now,  if  q    =.9,  then  n=.  991 
while  if  q.  =  .1    then  p,  =  .919  .    In  either  case,  E(pJ  \s  given  by    .975 
so  that  in  the  first  case  (p   =  .9) ,  b(p_)  =  -  .016  and  in  the  second  case  • 
b(pj  =  +  .066.    More  cases  are  treated  in  the  next  section. 

V 

3 .    Numerical  Comparisons 

To  gain  further  insight  into  the  results  of  the  last  section  as  well  as 
to  compare  these  results  with  those  previously  obtained  in  [2]  and  [3], 
it  was  decided-to  examine  special  cases  numerically.    For  this  purpose 
the  examples  documented  in  [2]  were  used.    Such  examples  allegedly 
cover  a  wide  variety  of  cases  that  are  of  practical  significance.    The  tabu- 
lated results  may  be  found  in  the  appendix,  Section  5.    An  example  is  de- 
fined by  specifying  the  parameters  K,  p   ,  q. ,  q9,   . ..,  q*  .    Nine  such 

i  b 

specifications  are  given.    However,  in  each  example,  the  parameters 

a  ,  a    ,   . . .,  a    as  well  as    N    are  further  specified  to  provide  fifteen 

cases  in  all.    In  reality,  then,  135  examples  are  treated  in  the  appendix. 

For  each  of  these  examples    p-,  the  moments  of  each  of  the  three  estimators, 

Po/  Pr  i  Pc  are  recorded  as  well  as  the  bias  in  each  case.    In  addition, 

ODD 

the  value  of   p    determined  by  computer  simulations  in. [2]  is  given  for 

each  example . 

We  previously  remarked  in   Section  1  that  p      ,  the  conditional  mean  of 

the  true  current  reliability  is,  prior  to  the  experiment,  a  random  variable. 

* 
Even  after  the  experimental  results  are  known,  moreover,  the  value  of  p 

still  cannot  be  determined  because  of  the  unknown  parameters    p   ,  q  , . ..  ,<%.„. 

0        1  K. 

which  enter  explicitly  in  its  formula.    It  is  shown  in  [3]  that  the  variance 

* 
of  p      converges  to  zero  as    N    becomes  infinite.    Hence,  for  large    N, 

°  * 

the  values  of  p     (whatever  the  experimental  outcomes)    and  m-  /  its  mean , 

should  not  be  significantly  different.    The  tables  of   Section  5  show  that 

10 


these  two  quantities  differ  by  very  little  even  for  moderate,  values  of   N-<— 
at  least  for  the  examples  treated.    What  this  means,  of  course,  is  that 

any  estimator   we  choose  for  estimating    p. ,  a  parameter,  can  effectively 

* 
be  used  also  as  a  predictor  for  p     ,  a  random  variable.   ' 

o 

As  for  the  maximum  likelihood  estimator,  p   ,  the  tables  reveal  that 
the  bias  is  positive  in  practically  every  case  investigated.    In  some  cases, 
Examples  3  through  8,  the  amount  of  positive  bias  is  serous  enough  to 
make  its  use  doubtful.    Of  course,  we  are  speaking  hefe  |||i}y  of  unbiased- 
ness  as  a  criterion  for  choice.    Recalling  the  original  problem,  one  is 
attempting  to  take  credit  for  corrective  action  in  updating  reliability  over 
the  initial  state  of  nature  p    .    Positive  bias  indicates  a  tendency  to  take 
more  credit  than  is  due  and  such  optimism  can  be  very  misleading  as  to 
the  potential  worth  of  such  a  testing  program.    From  this  point  of  view,  p 
has  little  to  offer  the  experimenter.    The  result  is  not  too  surprising  since 


maximum  likelihood  estimators  tend  to  be  biased.    Moreover,  it  is  difficult 
to  justify  the  maximum  likelihood  criterion,  for  which  p    is  the  optimal 
choice,  as  one  to  adopt  in  the  present  circumstances  . 

As  far  as  unbiasedness  alone  is  concerned,  it  is  here  reiterated  as  in 
[3]  that  p    is  "effectively"  unbiased.    Indeed,  in  every  single  example 

D 

treated,  the  bias  of.p    is  zero  to  three  decimal  place    computation.    Not 

o 

one  case  arose  where  the  result  was  different  from  zero.    Such  a  situation 
is  not  surprising  for  large  values  of   N    as  brought  out  in  [3]  but  for  small 

/'- 

values  of   N   the  same  result  is  somewhat  surprising  and  helps  support  p 

o 

as  an  important  contender  for  use  as  an  estimator  for  p.  . 

It  was  anticipated  that  p_  would  underestimate   p,  since  it  was  defined 

b 

in  such  a  way  as  to  have  this  property.    The  amount  of  bias  is  somewhat 
serious  in  several  of  the  examples  (notably  1,  2,  4,  and  8)  .    As  remarked 
previously,  the  price  for  conservatism  may  be  too  high.    Certainly  we  wish 
to  avoid  overestimating;  at  the  same  time  we  should  not  want  to  be  unduly 
severe  so  that  we  certainly  wish  to  take  credit  for  corrective  action  when 
such  credit  is  due . 

11 


As  a  final  remark  we  note  the  following  interesting  result  in  the 
examples.   As  the  a.'s  decrease ,  it  is  noted  that  for  fix^ed    N,  the  bias 
approaches  zero  (from  either  side)  .    This  means  that  as  our  ability  to 
remove  the  cause  of  a  detected  failure  decreases  so  too  our  tendency 
to  overestimate  (or  underestimate,  as  the  case  may  be)  decreases . 

4  .    Topics  for  Further  Study 

It  is  concluded  that  the  matter  of  unbiasedness  for  the  model  treated 
in  this  report  is  settled.    P      is  preferable  to  the  three  estimators  ex- 
amined and  certainly    p      should  be  rejected  on  this  basis  ..   However  , 
als  previously  remarked,  unbiasedness  is  but  one  criterion.    It  is  well 
known  that  a  biased  estimator  is  preferable  over  an  unbiased  one  if  the 
variance  of  the  former  is  sufficiently  smaller  than  that  of  the  latter.  This 
suggests  adopting  mean  Squared  error  as  a  criterion  and  comparing  p   , 

p_  and  pc  on  this  basis . 
o  b 

In  [3]  the  .variance  for   p      has  been  derived  and  is  given  explicitly 

b 

at  least  up  to  higher  order  terms  .    No  such  expression  is  yet  available 
for  p    although  some  (umpublished)  computer  simulations  carried  out  in  con- 
nection  with  [3]  indicate  that  the  variance  of   p      dec^eases  rapidly  with 
N.    Clearly,  the  variance  for   p      can  be  written  down' along  the  lines  of 

3  ;,    ■     ^ 

the  first  moment  as  derived  in  Section  2  although  the  algebra  involved 
may  be  somewhat  unwiedly.    In  any  event,  numerical  values  can  certainly 
be  obtained  for  such  as  the  examples  treated  in  this  report.    With  such  tools 
at  hand,  the  three  estimators  could  then  be  compared  on  a  mean  squared 
error  basis . 

Since    \i    is  a  well-defined  function  of  the  unknown  parameters  involves , 
another  problem  suitable  for  investigation  is  that  of  finding  a  Bayes  estimator 
for   m-  .    Some  a  priori  assumptions  about    p   ,  q. ,   .'..,  q    would  of  course 

.     O  1  K  %&..'; 

have  to  be  made  and  the  results  judged  accordingly.  Such  an  estimator  should 
then  be  compated  with  the  other  candidates  as  to  unbiasedness,  mean  squared 
error,  etc. 

12 


As  yet,  little  progress  has  been  made  within  this  model  in  the 
direction  of  confidence  interval  estimation.    In  no  small  part,  this  is 
due  to  the  complete  lack  of  distribution  theory  with  regard  to  the  esti- 
mators treated.    It  would  be  most  desirable  to  study  the  problem  of  find- 
ing a  lower  one-sided  confidence  interval  for    p,   .    Even  approximate  re- 
sults would  be  beneficial  to  the  present  state  of  the  art.  • 

Still  another  problem  worthy  of  investigation  is  a  re-examination 
of  the  model  itself.    In  spite  of  its  reasonableness,  same  aspects  of  the 
model  are  somewhat  confining.    Most  notably,  the  ma,|^]|fpf  allowing 
failures  to  accumulate  until  all    N    tests  are  performed  may  be  intolerable 
in  some  practical  situations  .    It  may  be  far  more  reasonable  to  stop  test- 
ing as  soon  as  a. failure  is  observed,  take  the  necessary  corrective  action, 
then  proceed  as  before  until  the  next  failure  occurs .    Such  a  program  of 
testing  would  thus  involve  several  stages.    In  a  given  stage,  the  sample 
obtained  would  be  a  sample  from  a  geometric  distribution  (having  observed 
Bernoulli  trials  to  first  failure)  but  the  parameter  changes  from  one  stage 
to  the  next  if  corrective  action  is  successful.    Again,  various  quantities 
related  to  the  growth  in  reliability  could  be  examined  under  this  model . 

A  model  similar  to  that  just  outlined  is  presented  in  a  report  [1] 
which  appeared  recently  in  the  literature .    A  brief  examination  of  this  re- 
port reveals  several  shortcomings  which  will  need  to  be  overcome  before 
the  usefulness  of  the  results  can  be  assessed.    In  any  case,  the  work  pre- 
sented there  should  be  more  closely  examined  if  further  study  along  the 
lines  presented  above  is  pursued. 


-■■'.: 


5 .    Appendix 

The  tables  to  follow  summarize  the  numerical  results  which  are  analyzed 
in  Section  3 .    The  tables  are  self-explanatory  with  all  of  the  notation  con- 
sistent with  that  previously  adopted  in  this  report.    The  examples  were 

limited  to  those  available  in  [2]  in  order  to  avoid  computer  simulations 

* 
needed  for  evaluating    p    .    Otherwise,  any  number  of  further  examples  may 

be  defined  as  in  the  tables  and  the  corresponding  entries  easily  computed. 
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EXAMPLE 


K  =  9  PQ    =    .10 


%**♦ 


ql    =    q2  =  *  * '  =  q9  =  *10 


,^Y 

a.     =    1.0 
i 

a      a     0.8 

a.      =    0.6 
l 

N 

5 

10 

25 

50 

100 

5 

10 

25      50    100 

5 

10 

25 

50    100 

* 
po 

.434 

..649 

.928 

.993 

1.00 

,370 

.549 

.748 

.815 

.820 

.313 

.431 

.594 

.635 

.64( 

M- 

.469 

.686 

.935 

.995 

1.00 

.395 

.569 

.768 

.816 

.820 

.321 

.452 

.601 

.637 

.646 

E(P3) 

.786 

.836 

.937 

.988 

1.00 

.649 

.689 

.769 

.810 

.820 

.511 

.542 

.602 

.633 

.646 

b(P3) 

.317 

.150 

.002 

.007 

0 

.254 

.120 

.001 

# 
.006 

0 

.190 

.090 

.001 

.004 

0 

E(p5) 

.469 

.686 

.935 

.995 

1.00 

.395 

.569 

.768 

.816 

.820 

.321 

.452 

.601 

.637 

.64( 

b(p5) 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

E(P6) 

.406 

.651 

.928 

.995 

1.00 

.348 

.541 

.76- 

.816 

.820 

.286 

.431 

.597 

.637 

.646 

b(pfi) 

.06  c 

.035 

It 

.007 

0 

0 

F 
.047 

1 
.028 

.005 

0 

0 

F 

,035 

It 
.021 

.004 

0 

0 

#  -  indicates  negative  value 


TABLE       1 


1'4 


EXAMPLE 


K  =   10        p      =    .10 

qi  =  q2=.20         q3=q4=q5=  .10 


q6=  V'iSli)   =     -°4 


I 


a.     ■    1.0 
i 

a.     s    0.8 
i 

a       =0.6 

N 

5 

10 

25 

50 

100 

5 

10 

25      50    100 

5 

10 

25 

50    100 

* 
Po 

.519 

.718 

.902 

.974 

.997 

.451 

.589 

.747 

.796 

.816 

.34$ 

.468 

.584 

.622 

.63$ 

H 

.529 

.719 

.905 

.972 

.997 

.443 

.596, 

744 

.798 

.817 

.35/ 

.472 

.583 

.623 

.636 

E(P3) 

.80' 

.856 

.936 

.976 

.994 

.663 

.705 

.769 

.800 

.815 

.522 

.553 

.602 

,625 

,636 

b(p3) 

.275 

.136 

.031 

.004 

1 
.003 

.220 

.109 

.025 

.002 

# 
.002 

.165 

.081 

.019 

,002 

002 

E(P5) 

,529 

.719 

,905 

.972 

.997 

.443 

.596 

.744 

.798 

.817 

.357 

.472 

.583 

,623 

.638 

b(p5) 

0 

0 

0 

0 

6 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

E(P6) 

.469 

.692 

.899 

.971 

.996 

.396 

.573 

.739 

.797 

.817 

.322 

.456 

.579 

623 

638 

E>(P6) 

.060 

.027 

I       f 
.006|.001 

.001 

F  ■ 

.047 

.023L005 

.001 

0 

.035 

^016 

L004 

0 

0 

#■-  indicates  negative  value 


TABLE       2 


1<5 


;■• 


EXAMPLE 


K  =  100      p      =      .10 
o 

Ql=    q2=-"=    qi00=    -009 


a.    =    1.0 
i 

ai    a     0.8 

a       =0.6 

N 

5 

10 

25 

50 

100 

5 

10 

25      50    100 

5 

10 

25 

50    100 

* 

Po 

.139 

.176 

282 

.426 

.636 

.132 

.162 

.245 

.358 

.533 

.124 

.146 

,210 

.297 

,42J 

M- 

.140 

.178 

,282 

.427 

.636 

.132 

.162 

.246 

.362 

.528 

.124 

.147 

.209 

.296 

.421 

E(pa) 

.713 

.703 

,719 

.753 

.814 

.590 

.583 

.595 

.623 

.671 

.468 

.462 

.471 

.492 

.52S 

b(P3) 

.573 

.525 

,437 

.326 

.178 

.458 

.421 

.349 

.261 

.143 

.344 

.315 

.262 

.196 

.1Q7 

E(P5) 

,140 

.178 

,282 

.427 

.636 

.132 

.162 

.246 

.362 

.528 

,124 

.147 

.209 

.296 

.421 

b(P5) 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

E(p6) 

132 

.170 

276 

,422 

.632 

.126 

.156 

.240 

.358 

.526 

119 

.142 

.205 

.,293 

,419 

b(P6) 

#      • 
008 

.008 

006 

# 
r005 

# 
.004 

# 
.006 

# 
.006 

.006 

.004 

# 
.002 

# 
005 

# 
.005 

# 
.004 

,003 

# 
f002 

#  -  indicates  negative  value 


TABLE 


16. 


EXAMPLE 


K  =  100       PQ    -    .10 


q  =q     =  .20       Qa^q.-q'     =  .10       q     =  .  . .  =  q ,     =  .004     q    =  . 


l3   ^4  ^5 


10 


'11 


qioo=-002    ? 


a 

=    1.0 
l 

a      a    0.8 

8i     S    ° 

.6 

N 

5 

10 

25 

50 

100 

5 

10 

25      50    100 

5 

10 

25 

50    100 

* 

.502 

.650 

.77.7 

.819 

.840 

.426 

.545 

.653 

.676 

.691 

.337 

.437 

.515 

,531  ,544 

n 

.494 

.657^ 

.788 

.819 

.839 

.415 

.545 

.650 

.675 

.691 

,336 

.434 

.513 

,532 

.54' 

E(pa) 

.796 

.841 

.905 

.928 

.936 

.657 

.693 

.744 

.7.62 

.769 

,518 

.545 

.583 

,597 

602 

b(P3) 

.302 

.184 

.117 

.109 

.097 

.242 

.148 

.094 

.087 

.078 

,182 

.111 

.070 

,065 

058 

E(P5) 

.494 

.6S7 

,788 

.819 

.839 

.415 

.545 

.650 

.672 

.691 

,336 

.434 

.513 

,532 

544 

b(P5) 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

E(P6) 

.441 

.634 

.785 

.819 

.839 

.373 

.527 

.648 

.672 

.691 

,305 

.420 

.511 

531 

.544 

b(pg) 

# 

.053 

— — 

# 
.023 

.002 

0 

0 

.042 

# 

.018 

r 

.002 

0 

0 

# 
.031 

# 
.014 

.002 

# 
001 

0 

#  -  Indicates  negative  value 


TABLE      4 


17. 


s 


EXAMPLE 


K  =  100       p      =  .10 
q1=q2Q3  =  .10    q4=  .003    q5 


i 


...=q10=,020     qiV^.-^ioo 


=    .005 


■ 


a.     =    1.0 
i 

a.     -     0.8 
l 

aA     =0.6 

N 

5 

10 

25 

50 

100 

5 

10 

25      50    100 

5 

10 

25 

50    100 

* 
Po 

.246 

.351 

.503 

.599 

.714 

.213 

.304 

.418 

.499 

.586 

.184 

.245 

.338 

.397. 

.464 

n 

.246 

^339 

.479 

.575 

.682 

.216 

.292 

.403 

.480 

.566 

.187 

.244 

.328 

.385 

566 

E(P3) 

.718 

.728 

.770 

.805 

.840 

.595 

.603 

.636 

.664 

.692 

.471 

.477 

.502 

.523 

692 

b(P3) 

.472 

.389 

.291 

.230 

.158 

.379 

.311 

.233 

.184 

.126 

.284 

.233 

.174 

.138 

126 

E(p5) 

.246 

.339 

,479 

.575 

.682 

.216 

.292 

.403 

.480 

.566 

.187 

.244 

.328 

,385, 

566 

b(p5) 

0 

0 

0 

0 

•o 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

E(P6) 

.221 

.324 

.474 

.572 

.681 

.197 

.279 

.399 

.478 

.564 

473 

.234 

.324 

,383. 

564 

Kp6) 

# 
.025 

# 
.015 

# 

.005 

# 
.003 

.001 

# 
.019 

# 
.013 

# 
.004 

# 
.002 

# 
.002 

# 
.014 

# 
.010 

# 
.004 

# 
,002, 

#' 

m 

**£ 


<y"   >  v,. 


.'  : 


#  -  indicates  negative  value 


TABLE 


.18 


EXAMPLE 


K=    1000   PQ    =  .10 

Ql   =  q2  =  '  •  *  =  qi000  =     -0009 


a.     a    1.0 
l 

a,     s     0.8 
l 

a.      ■    0.6 
l 

N 

5 

10 

25 

50 

100 

5 

10 

25      50    100 

5 

10 

25 

50    100 

* 
Po 

.104 

.1-98 

.120 

.140 

.177 

.103 

.106 

.116 

.132 

.162 

,102 

.105 

.112 

.124 

.147 

J* 

.104 

.108 

.120 

.140 

.177 

.103 

.106 

.116 

.132 

.162 

.102 

.105 

.112 

.124 

.146 

E(P3) 

.70* 

.688 

.680 

.681 

.689 

.585 

.570 

.564 

.565 

.571 

.46' 

.45: 

.448 

.449 

.453 

b(P3) 

.602 

.580 

.560 

.541 

.512 

.482 

.464 

.448 

.433 

.409 

•  36! 

.34* 

.336 

.325 

.307 

E(P5) 

.104 

.108 

,120 

.140 

.177 

.103 

.106 

.116 

.132 

.162 

.102 

.105 

.112 

.124 

.146 

b(p5) 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

E(P6) 

.103 

.107 

.119 

.139 

.177 

.103 

.106 

.115 

.131 

.161 

.102 

.104 

.112 

.123 

.146 

Mp6) 

# 
.001 

# 
.001 

.001 

.001 

0 

0 

0 

.001 

# 
.001 

# 
.001 

0 

# 
.001 

0 

# 
.001 

0 

#  -  indicates  negative  value 


TABLE     6 
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s 


EXAMPLE 


K  =  100      p      =     .60 
o 

qi =  q2  •  v  =  qioo  =  -004 


a.     ■    1.0 
i 

a.     a     0.8 
l 

a      =0.6 

N 

5 

10 

25 

50 

100 

5 

10 

25      50    100 

5 

10 

25 

50    100 

* 
Po 

<608 

.615 

.639 

.672 

.733 

.606 

.613 

.631 

.660 

.706 

.60S 

.60S 

.623 

.643 

.68( 

V- 

,608 

.616 

.638 

.673 

.732 

.606 

.613 

.631 

.658 

.706 

.605 

.£09 

.623 

.644 

.67< 

E(P3) 

,871 

.864 

• 

.865 

.872 

.886 

.816 

.811 

.812 

.817 

.829 

762 

1  fe  Si 

.758 

.759 

.763 

,772 

b(p3) 

,263 

.248 

.227 

.199 

.154 

.210 

.198 

.181 

.159 

.123 

.157 

.149 

.136 

.119 

,093 

E(P5) 

608 

.616 

.638 

.673 

.732 

.606 

.613 

.631 

.658 

.706 

.605 

.609 

.623 

.£.44 

679 

b(p5) 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0     i 

0 

0 

E(P6) 

.606 

.614 

.637 

.671 

.731 

.605 

.611 

.629 

.657 

.705 

,604 

.609 

.622 

.£43 

679 

b(p6) 

# 
.002 

# 
.002 

# 
.001 

# 
.002 

It 
.001 

# 
.001 

# 
.002 

ft 

.002 

# 
.001 

.001 

# 
,001 

0 

.001 

# 
k001 

# 
0 

#  -  indicates  negative  value 


TABLE 
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EXAMPLE 


I 

B 


K  =  81        PQ    =     -60 


I 


qj=  .10    ,q2=  ...=  q6  =    .03    q?  =  . .  .=  q^  =  .004    q^  =  .  ..=  qgl=  .001 


. 


ajt    s    1.0 

a.     si     0.8 

i 

a      =0.6 

N 

5 

10 

25 

50 

100 

5 

10 

25      50    100 

5 

10 

25 

50    100 

* 

po 

.667 

.710 

.774 

.836 

.879 

.655 

.685 

.741 

.790 

.824 

,644 

.663 

.709 

.742 

,769 

^ 

.66^ 

.709 

.784 

.837 

.881 

.651 

.687 

.747 

.790 

.825 

,639 

.665 

.710 

.742 

,768 

E(p'a» 

.882 

.887 

.908 

.928 

.946 

.826 

.830 

.846 

.862 

.877 

.770 

.772 

.785 

.797 

,808 

b(p3) 

.219 

.178 

.124 

.091 

.065 

.175 

.143 

.099 

.072 

.0  52 

.131 

.107 

.075 

.055 

,040 

E(P5) 

.664 

.709 

.784 

.837 

.881 

.651 

.687 

.747 

.790 

.825 

j 
.636 

.665 

.710 

.742 

.76E 

b(p5) 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

-2> 

0 

0 

0 

0 

E(P6) 

.65c 

.701 

.780 

.836 

.880 

.643 

.681 

.74< 

.78$ 

.824 

.632 

.661 

.708 

.742 

.766 

b(p6) 

# 
.011 

# 
.008 

.004 

* 
.001 

# 
.001 

# 
.008 

# 
.006 

5" 

.00  c 

# 
.001 

# 
.001 

# 
k007 

# 
.00< 

It 
.002 

0 

1 

o 

#  -  indicates  negative  value 
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EXAMPLE 


K  s  100       p      =     .90 
o 

qi =  •  •  •  qioo  =  -001 


a.     s    i.o 
i 

a.     s    0.8 
i 

a.      =    0.6 
i 

N 

5 

10 

25 

50 

100 

5 

10 

25      50    100 

5 

10 

25 

50    100 

* 
Po 

.900 

.901 

.902 

.905 

.910 

.900 

.901 

.902 

.904 

.907 

.900 

.901 

.902 

.903 

.90! 

H 

.900 

.901 

.902 

.905 

.910 

.900 

.901 

.902 

.904 

.908 

.900 

.901 

.901 

.903 

,906 

E(P3) 

.967 

.965 

.965 

.965 

.966 

.954 

.952 

.952 

.952 

.953 

.940 

.939 

.939 

.939 

939 

b(P3) 

.067 

.064 

.063 

.060 

.056 

.054 

.051 

.050 

.048 

.045 

,040 

.038 

.038 

.036 

,033 

E(P5) 

.900 

.901 

.902 

.905 

.910 

.900 

.901 

.902 

.904 

.908 

,900 

.901 

.901 

,903 

,906 

b(p5) 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

E(P6) 

.900 

.901 

.902 

.905 

.090 

.900 

.901 

.902 

.904 

.908 

900 

.901 

.901 

903 

906 

b(P6) 

0 



0 

0 

0 

# 
.001 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

#  -  indicates  negative  value 
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